Skip to content

format: a CSV can be as narrow or as wide as the thing you are testing - #47

Merged
donislawdev merged 1 commit into
mainfrom
format/csv-columns
Sep 3, 2026
Merged

format: a CSV can be as narrow or as wide as the thing you are testing#47
donislawdev merged 1 commit into
mainfrom
format/csv-columns

Conversation

@donislawdev

Copy link
Copy Markdown
Owner

Adds columns to the csv format, from 2 to 32768, six by default.

Nothing anybody already generates moves by a byte. Six is the six columns
this tool has always written, and the three pinned csv hashes are unchanged -
which is the whole claim this setting makes. A fourth case pins a width of
seventeen from the day it exists.

Which columns go, and which stay

Fewer than six drops them from the middle. id stays first and
description stays last, because description is the field the closing row
stretches to reach the exact size that was ordered - put it anywhere else and
the padding lands inside a record while the length stays perfect. More than six
adds field_7, field_8 and so on in front of it.

Column types stay out of this. They have their own place in the backlog, and
half a type system built here would be the harder half to take back.

The ceiling was measured, not recalled

LibreOffice Calc headless, tables written by hand, three readers asked at once:

columns Python csv our checker LibreOffice Calc
64, 256, 1024, 4096 OK OK whole
16384 OK OK whole
16385 OK OK kept 16384 - the last column is gone, with no warning
32768 OK OK kept 16384, in 3 s

So the ceiling sits deliberately above that line. Building the set either
side of a boundary is what this tool is for, and stopping at 16384 would offer
the last table that survives and not the first that does not.

Copying the 64 that xlsx uses was rejected: that number is justified by a
sheet built in memory and by the width a person looks at, and neither applies to
a format that is streamed. Borrowing someone else's reason is worse than having
none.

The floor is two, for three reasons

A table of one column carries no separator anywhere, so a file written with the
wrong one is byte for byte a file written with the right one - our own
structural check refuses such a file for exactly that reason. It would also
leave delimiter doing nothing, which this project treats as a refusal rather
than a silence. And one column of values is a txt file, which this tool
already writes.

The floor moves with the width, as it already does for row endings and quoting:
36 B at two columns, 115 B at six, 5017 B at 256, 709793 B at the
ceiling. Twelve widths measured with the binary, all distinct.

Verified

  • 150 files swept - ten widths, three quote styles, five sizes from each
    floor upward. Exact size, exact width under both Go's encoding/csv and
    Python's module, and the manifest agreeing with the file.
  • The structural checker is now told the width and asserts it. That is
    stronger than what it did before: counting can only ask whether the rows agree
    with each other, and a table that wrote five columns where six were ordered
    agrees with itself perfectly at exactly the right length. Negative controls
    both ways.
  • 19 of 19 mutations caught.

Two guards were wrong before this

The one written for quote_style yesterday counted separators as "minus five",
because six fields need five. That was right only because it never changed the
width - it asks the plan now. And the manifest guard never checked the column
count at all.

Full suite, preflight --quick (12 of 12, including the three CI-only checks)
green locally. Two failures were caught by preflight rather than by the targeted
runs: parseDialect crossed the branching band and needed splitting, and the
changelog entry had created a second ### Added heading.

🤖 Generated with Claude Code

Adds columns to the csv format, from 2 to 32768, six by default. Six is
the six columns this tool has always written, so nothing anybody already
generates moves by a byte - the three pinned csv hashes are unchanged and
that is the whole claim this setting makes.

Fewer than six drops them from the MIDDLE. id stays first and description
stays last, because description is the field the closing row stretches to
reach the exact size that was ordered - put it anywhere else and the
padding lands inside a record while the length stays perfect. More than
six adds field_7, field_8 and so on in front of it. Column TYPES stay out
of this: they have their own place in the backlog, and half a type system
built here would be the harder half to take back.

The ceiling was chosen by measuring other people's readers rather than
from memory. LibreOffice Calc headless, tables written by hand: 16384
columns come back whole, 16385 comes back with 16384 and the last column
gone without a word, 32768 comes back as 16384 in three seconds. Python's
csv module and our own checker take all of them.

So the ceiling sits deliberately ABOVE that line. Building the set either
side of a boundary is what this tool is for, and stopping at 16384 would
offer the last table that survives and not the first that does not.
Copying the 64 that xlsx uses was rejected: that number is justified by a
sheet built in memory and by the width a person looks at, and neither
applies to a format that is streamed. Borrowing someone else's reason is
worse than having none.

The floor is two, and for three reasons rather than one. A table of one
column carries no separator anywhere, so a file written with the wrong one
is byte for byte a file written with the right one - our own structural
check refuses such a file for exactly that reason. It would leave the
delimiter setting doing nothing, which this project treats as a refusal
rather than a silence. And one column of values is a txt file, which this
tool already writes.

The floor moves with the width, as it already does for row endings and
quoting: 36 B at two columns, 115 B at six, 5017 B at 256, 709793 B at the
ceiling. Measured with the binary, twelve widths, all distinct.

The structural checker is now told the width and asserts it, which is
stronger than what it did before. Counting can only ask whether the rows
agree with each other, and a table that wrote five columns where six were
ordered agrees with itself perfectly at exactly the right length.

Verified: 150 files swept over ten widths, three quote styles and five
sizes from each floor upward - exact size, exact width under both Go's
encoding/csv and Python's module, and the manifest agreeing with the file.
19 of 19 mutations caught.

Two guards were wrong before this and are fixed here. The one written for
quote_style counted separators as "minus five", which was right only
because it never changed the width - it asks the plan now. And the
manifest guard never checked the column count at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@donislawdev
donislawdev merged commit 4063fca into main Sep 3, 2026
18 checks passed
@donislawdev
donislawdev deleted the format/csv-columns branch September 3, 2026 17:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant